Providing image content (e.g., media content or non-media content) from a mobile device to a remote display for presentment via wireless display technology is becoming increasingly popular. Supporting such wireless displays is difficult for small form factor mobile devices and may become even more so as higher resolution wireless displays (e.g., retinal quality displays) are implemented and as the mobile devices become thinner and lighter with tighter thermal constraints. Also, the wireless connection between the devices may become more and more congested as more devices are put in use. Furthermore, such wireless display support and other types of wireless input/output (IO) may increase as wireless usages including productivity applications, gaming, conferencing, and other difficult workloads become more mainstream.
In such environments, user experience including battery life, image quality, end-to-end latency, and the like will be difficult to maintain and improve. For example, in non-media environments (e.g., productivity, web browsing, and the like), human perception is acutely latency-biased. In such non-media environments, perceptual responsiveness (e.g. responsiveness to touch or other inputs) is important as is energy efficiency (e.g., to increase battery life). However, current wireless display solutions and/or specifications such as Wireless Display (WiDi) and/or Miracast may be optimized for media workloads such as video playback, photo viewing, and the like. For such media workloads, a constant frame rate may be advantageous and current solutions may seek to provide a constant frame rate at the expense of latency and/or energy efficiency problems.
More specifically, delays and bandwidth variations are inherent problems with wireless transports such as WiFi and WiGig. For example, delays on the order of 10 milliseconds (ms) may occur relatively frequently and even longer delays are not uncommon. Furthermore, even wired interfaces such as Ethernet and USB Bulk may provide non-trivial delays and bandwidth variations due to the shared nature of the transport. Also, use of non-deterministic transport in the context of an isochronous flow (e.g., wireless display) may accentuate such problems and/or provide additional problems. For example, if a source and target device in a wireless display setting or system operate natively at 60 Hz, a delayed frame (or failure to drop such a delayed frame) may make the target device lag one frame (e.g., 16.67 ms) behind the source device. Multiple such events may cause a significant lag with no way to recover without dropping frames. However, many wireless display encoding standards such the H.264/Advanced Video Coding (AVC) standard require the target device to decode all frames. Otherwise, the target device may miss data required for the decoding of subsequent frames. Furthermore, typical decoders may not be capable of decoding multiple frames per frame-time (e.g., typical decoders may decode a maximum of two frames per 16.67 ms interval), limiting the ability of the target device to catch up on significant delays. In any event, such delayed frames may not be displayed (e.g., at a 60 Hz target device) and therefore may not provide direct perceived user value.
As discussed, current wireless display solutions may focus on media workloads and some implementations may attempt to address the problem of delayed frames by implementing a large jitter buffer, which may add a fixed latency between the source device and the target device. Such a solution may be acceptable for media content (e.g., using a 100-200 ms buffer) where end-to-end latency is not a issue. However, such jitter buffer solutions are not typically acceptable for non-media and/or interactive workloads such as office productivity applications, gaming, web browsing, and many other applications that are typical in touch-screen interactions.
As such, existing techniques do not provide low latency and energy efficient wireless transmission of image content for remote display of non-media and/or interactive content. Such problems may become critical as the desire to present high quality image content that is responsive and can be generated with low power becomes more widespread.